Gradient Regularized Contrastive Learning for Continual Domain Adaptation

Human beings can quickly adapt to environmental changes by leveraging learning experience. However, adapting deep neural networks dynamic environments machine algorithms remains a challenge. To better understand this issue, we study the problem of continual domain adaptation, where model is presented with labelled source and sequence unlabelled target domains. The obstacles in are both shift catastrophic forgetting. We propose Gradient Regularized Contrastive Learning (GRCL) solve obstacles. At core our method, gradient regularization plays two key roles: (1) enforcing not harm discriminative ability features which can, turn, benefit adaptation domains; (2) constraining increase classification loss on old domains, enables preserve performance domains when an in-coming domain. Experiments Digits, DomainNet Office-Caltech benchmarks demonstrate strong approach compared state-of-the-art.